Reuse-Aware Management for Last-Level Caches

Variability in generational behavior of cache blocks is a key challenge for cache management policies that aim to identify dead blocks as early and as accurately as possible to maximize cache efficiency. Existing management policies are limited by the metrics they use to identify dead blocks, leading to low coverage and/or low accuracy in the face of variability. In response, we introduce a new metric – live distance – that uses the stack distance to learn the temporal reuse characteristics of cache blocks. We further introduce Leeway, a new dead block predictor that leverages live distance to enable dead block predictions that are robust to variation in generational behavior. Based on the reuse characteristics of application’s cache blocks, Leeway classifies its behavior as streaming or reuse and dynamically selects an appropriate cache management policy.